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DETAILED ACTION 

1 . In view of the Appeal Brief filed on 1 1/04/2009, PROSECUTION IS HEREBY 
REOPENED. New grounds of rejection are set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1 ) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1 .1 13 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41 .20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 

/Bhavesh M Mehta/ 

Supervisory Patent Examiner, Art Unit 2624. 

2. The Examiner has reopened prosecution in order to address one of the 
Applicant's main arguments throughout the appeal brief wherein the Applicant alleges 
and repeatedly argues that the plan view image is two dimensional as is understood by 
"any one of ordinary skill in the art" [e.g. see "This section describes Appellant's ..." on 
page 12 of the appeal brief filed on 11/04/2009]. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not Identically disclosed or described as set 
forth In section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
Invention was made to a person having ordinary skill In the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the Invention was made. 

4. Claims 1 , 3-7, 9,12,15,1 9-22, 23, 25-26, 29-31 , 32, 34-37 and 39 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Mahbub (US 2002/0050924 A1, as 
applied in previous Office Action) in view of Beymer ("Person Counting Using Stereo" - 
2000 IEEE, as applied in previous Office Action). 

Re Claim 23: Mahbub discloses a visual-based recognition system / occupant sensing 
system comprising a visual sensor / CCD stereo vision system for capturing depth data / 
3D image relating to distance for at least a pixel of an image of an object / seating area 
of a vehicle with possible occupant, which is not required to be inside of a subject / 
occupant (see Mahbub, [0045], lines 2-4, [0051], lines 1-3, [0081], lines 1-4, [0088], the 
object is not inside the subject / occupant), said depth data / 3D image comprising 
information relating to a distance / distance from said visual sensor / CCD stereo vision 
system to a portion of said object / seating area of vehicle with possible occupant visible 
at said pixel (see Mahbub, [0045], lines 2-4, [0051], lines 1-3, [0081], lines 1-4, [0088]), 
said visual sensor / CCD stereo vision system comprising an emitter and sensor of light 
(a CCD camera system inherently has an emitter and sensor of light), wherein said light 
is selected from the group of electromagnetic radiation consisting of visible light, 
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infrared light, and ultraviolet light (a CCD camera system inherently operates under 
visible light) and wherein said capturing of said depth data does not require special 
behavior from one of said object and said subject (Mahbub doesn't require any special 
behavior from the object or subject / occupant when producing the 3D image); a plan- 
view image generator / segmentation of a scene for generating a plan-view image / 
segmented 3D image [3D image essentially possesses multiple layers of 2D images at 
a variety of depths] based on said depth data / 3D image components relating to 
distance (see Mahbub, [0051], lines 1-3, [0059], [0061], lines 3-6, [0088], the 3D image 
is segmented to remove background clutter using thresholding means with the 3D X Y 
and Z components which relate to the distance of the object to the imaging camera 
system), wherein said generating of said plan-view image / segmented 3D image 
includes generating said plan-view image as if said object / seating area of a vehicle 
with possible occupant were viewed from above / headliner above the rearview mirror 
(see Fig. 1b) and wherein generating other view images based on different orientations 
of said object other than from above is not required / at least one imaging device 
wherein the imaging device is the headliner above the rearview mirror (see Mahbub, 
Figs. 1b and 2, [0040]-[0042] and [0056], at least one imaging device of the 3-D imaging 
system provides image data and this 3-D imaging system can be located at a variety of 
locations [e.g. 1 . headliner, 2. pillar, 3. dashboard, the Examiner is considering only one 
is used because "at least one" is stated], Mahbub is suggesting different location 
alternatives from where the 3-D image data may be produced and therefore the 
Examiner is considering Mahbub's plan-view image to be the segmented 3D image from 
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the headliner location [headliner has a fixed overhead location and a fixed overhead 
orientation] because the headliner above the rear view mirror which is above the seating 
area provides the maximum field of view with minimal obstruction); a plan-view template 
generator / 2D image generator for generating a plan-view template / 2D XY, YZ, or ZX 
plane images based on said plan-view image / segmented 3D image (see Mahbub, 
[0088], the segmented 3D image is projected to a 2D data set which will be used to 
classify for specific scenarios [e.g. occupant present, occupant not present, etc.], 
Mahbub's plan-view template [2D XY, YZ, or ZX plane images] which are used for 
classifying are the different projection perspective views of the 3D {ROI} segmented 
image as discussed in paragraph [0088] and the claim doesn't state the type of 
transformation that could be used); and a classifier / robust classifier for making a 
decision concerning recognition of said object / distinguish between scenarios, wherein 
said classifier is trained to make said decision according to pre-configured parameters / 
2D features that were determined at least in part based on a class assigned to said 
plan-view template / 2D XY, YZ or ZX plane images (see Mahbub, [0081], lines 1-4, 
[0088], [01 11], the robust classifier trained with the 2D features [the different 2D 
features are represented by Central Moments, Normalized moments, invariant 
moments, perimeter, area, eccentricity, etc.] with respect to the 2D plane images 
classifies if there is an occupant in the seating area, occupant forward facing, occupant 
reverse facing, etc.). 

However Mahbub doesn't explicitly suggest that the plan-view image is two 
dimensional. 
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Beymer discloses generating a three-dimensional subset of pixels / 3D volume of 
interest (3D VOI) based on said depth data / Z field in the 3D X, Y, Z image using an 
overhead camera (see Beymer, abstract. Figs. 1 and 3, Sections 4 Tracking in 
occupancy maps and 4.1 Occupancy maps, the 3D reconstruction from stereo from an 
above head camera system allows a three-dimensional volume of interest VOI to be 
analyzed, last paragraph of Section 6, the processing is done on a computer 
processor), and generating a plan-view image / occupancy orthographic map based on 
said three-dimensional coordinates [X, Y and depth Z], wherein said plan-view image is 
a two-dimensional representation of said three-dimensional subset of pixels / 3D VOI 
(see Beymer, abstract. Figs. 1 and 3, Sections 4 Tracking in occupancy maps and 4.1 
Occupancy maps, the 3D VOI is projected to the orthographic map as seen as the 2D 
image in Fig. 3a where the entire area of the person is drawn as an enclosing circle and 
this occupancy orthographic map is used for tracking of the person), and extracting a 
plan-view template / gaussian mixture model [a type of transformation] from said plan- 
view image / occupancy orthographic map in order to allow a track classification to 
occur (see Beymer, Fig. 5, Section 4.2 and 5.2-5.3, the Gaussian mixture model image 
in Fig. 5 is extracted or generated using the occupancy orthographic map image in Fig. 
5 essentially to assist in tracking and classifying the presence of a human). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify Mahbub's system using Beymer's teaching by 
including the segmentation of 3D VOI to Mahbub's plan-view image [segmented 3D 
image] in order to improve the tracking of people by focusing the computation on the 
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heads and torsos of the people (see Beymer, abstract, Figs. 1 and 3, Sections 4 
Tracking in occupancy maps and 4.1 Occupancy maps). 

Re Claim 25: Mahbub further discloses wherein said visual sensor / CCD stereo vision 
system determines said depth data / 3D image relating to distance using stereopsis / 
stereo vision based on image correspondences (see Mahbub, [0045], lines 2-4, [0051], 
lines 1-3, [0081], lines 1-4, [0088]). Beymer also further discloses wherein said visual 
sensor / stereo head camera determines said depth data / Z field in the 3D X, Y, Z 
image [Z field relates to distance] using stereopsis / stereo vision based on image 
correspondences (see Beymer, abstract. Figs. 1 and 3, Section 3, Sections 4 Tracking 
in occupancy maps and 4.1 Occupancy maps, the 3D reconstruction from stereo from 
an above head camera system allows a three-dimensional volume of interest VOI to be 
analyzed). 

Re Claim 26: Mahbub further discloses said plan-view image generator comprises a 
pixel subset selector / ROI for selecting a subset of pixels of said image, wherein said 
pixel subset selector / ROI is operable to select said subset of pixels based on 
foreground segmentation / segmentation of scene (see Mahbub, [0059], the 
segmentation of a scene determines the region of interest ROI by removing and 
eliminating background clutter). Beymer also further discloses a pixel subset selector / 
volume of interest VOI for selecting a subset of pixels of said image, wherein said pixel 
subset selector / VOI is operable to select said subset of pixels based on foreground 
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segmentation / segmentation of scene by focusing on liead and upper torso of tlie 
shoppers and removing the shopping cart (see Beymer, abstract, Figs. 1 and 3, 
Sections 4 Tracking in occupancy maps and 4.1 Occupancy maps). 

Re Claim 29: Beymer further discloses wherein said plan-view image generator / 
occupancy orthographic map is operable to generate a three-dimensional point cloud 
based on said depth data / Z field in the 3D X, Y, Z image, wherein a point of said three- 
dimensional point cloud / 3D volume of interest (3D VOI) comprises a three-dimensional 
coordinate / X, Y, Z (see Beymer, abstract, Figs. 1 and 3, Sections 4 Tracking in 
occupancy maps and 4.1 Occupancy maps, the 3D reconstruction from stereo from an 
above head camera system allows a three-dimensional volume of interest VOI to be 
analyzed, last paragraph of Section 6, the processing is done on a computer 
processor). 

Re Claim 30: Beymer further discloses wherein said plan-view image generator / 
occupancy orthographic map is operable to divide said three-dimensional point cloud / 
3D volume of interest (3D VOI) into a plurality of slices / different buckets that a plan- 
view image may be generated for at least one slice of said plurality of slices (see 
Beymer, abstract. Figs. 1 and 3, Sections 4 Tracking in occupancy maps and 4.1 
Occupancy maps, the 3D VOI is generated by the different buckets, the different 
buckets of the 3D VOI are projected to the orthographic map as seen as the 2D image 
in Fig. 3a, the occupancy orthographic map image is produced by a type of accumulator 
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of pixels in the corresponding buckets, last paragraph of Section 6, the processing is 
done on a computer processor). 

Re Claim 31 : Beymer further discloses wherein said plan-view template generator / 
Gaussian mixture model is operable to extract a plan-view template / Gaussian mixture 
model image [a type of transformation] from at least two plan-view images / occupancy 
orthographic map [the occupancy orthographic map image is an accumulation of one or 
more layers of buckets] corresponding to different slices of said plurality of slices / 
different buckets, wherein said plan-view template / Gaussian mixture model [a type of 
transformation] comprises a transformation of at least said portion of said plan-view 
images, such that said plan-view template / Gaussian mixture model image is 
processed at said classifier / track classification (see Beymer, abstract. Figs. 1 and 3, 
Sections 4 Tracking in occupancy maps and 4.1 Occupancy maps, the different buckets 
of the 3D vol are projected to the occupancy orthographic map as seen as the 2D 
image in Fig. 3a, Fig. 5, Section 4.2 and 5.2-5.3, the Gaussian mixture model image in 
Fig. 5 is extracted or generated using the occupancy orthographic map image in Fig. 5 
essentially to assist in tracking and classifying the presence of a human). 

As to claim 1, the claim is the corresponding method claim to claim 23 
respectively. The discussions are addressed with regard to claim 23. Further Mahbub 
discloses receiving digital / CCD stereo vision system depth data / 3D image relating to 
distance (see Mahbub, [0045] at lines 2-4, [0051] at lines 1-3, [0081] at lines 1-4, 
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[0088]); wherein at least a portion of said plan-view image / segmented 3D image is 
transformed (Mahbub's plan-view template [2D XY, YZ, or ZX plane images] wliicli are 
used for classifying are the different projection perspective views of the 3D {ROI} 
segmented image as discussed in paragraph [0088], and the claim doesn't state the 
type of transformation that could be used); and processing that is executing on a 
computer system (see Mahbub, [0155]). 

As to claims 3-4, the claims are the corresponding method claims to claims 25-26 
respectively. The discussions are addressed with regard to claims 25-26. 

Re Claim 5: Beymer further discloses generating a three-dimensional point cloud of a 
subset of pixels / 3D volume of interest (3D VOI) based on said depth data / Z field in 
the 3D X, Y, Z image, wherein a point of said three-dimensional point cloud comprises a 
three-dimensional coordinate / X, Y, Z (see Beymer, abstract. Figs. 1 and 3, Sections 4 
Tracking in occupancy maps and 4.1 Occupancy maps, the 3D reconstruction from 
stereo from an above head camera system allows a three-dimensional volume of 
interest VOI to be analyzed); partitioning said three-dimensional point cloud into a 
plurality of vertically oriented bins / vertical buckets (see Beymer, Figs. 3a and 3b, 
Sections 4 Tracking in occupancy maps and 4.1 Occupancy maps, the 3D VOI is 
partitioned into vertical bins / buckets); and mapping at least a portion of points of said 
plurality of vertically oriented bins into at least one said plan-view image / occupancy 
orthographic map image based on said three-dimensional coordinates, wherein said 
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plan-view image Is a two-dimensional representation of said tliree-dimensional point 
cloud / 3D vol comprising at least one pixel corresponding to at least one vertically 
oriented bin of said plurality of vertically oriented bins / the different vertical buckets (see 
Beymer, abstract, Figs. 1 and 3, Sections 4 Tracking in occupancy maps and 4.1 
Occupancy maps, tlie different buckets of the 3D VOI are projected to the orthographic 
map as seen as the 2D image in Fig. 3a, the occupancy orthographic map image is 
produced by a type of accumulator of pixels in the corresponding buckets). 

Re Claim 6: Although Mahbub as modified by Beymer doesn't explicitly suggest 
receiving non-depth data for said pixel, and wherein said foreground segmentation is 
based at least in part on said non-depth data, the Examiner takes Official Notice that it 
would have been exceedingly obvious to one of ordinary skill in the art at the time of the 
invention to modify Mahbub to have the CCD camera include non-depth / color 
information to perform foreground segmentation because such limitations are well 
known and typical in the digital image color segmentation processing field in order to 
allow only data of interest to be considered in the processing of image data. 

Re Claim 7: Beymer further discloses dividing said three-dimensional point cloud / 3D 
volume of interest (3D VOI) into a plurality of slices / 2D XY layer for each depth Z, 
wherein said generating said plan-view image / occupancy orthographic map image is 
performed for at least one slice of said plurality of slices / 2D XY layer for each depth Z 
(see Beymer, abstract. Figs. 1 and 3, Sections 4 Tracking in occupancy maps and 4.1 
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Occupancy maps, the different budgets at multiple depth Z's of the 3D VOI are projected 
to the orthographic map as seen as the 2D image in Fig. 3a). 

Re Claim 9: Mahbub further discloses wherein said extracting said plan-view template 
from said plan-view image is based at least in part on object tracking (see Mahbub, 
[0059], the respective seating area is tracked and that respective area is extracted as a 
XY, YZ, or ZX plane image). Beymer also further discloses said extracting said plan- 
view template / Gaussian mixture model [a type of transformation] from said plan-view 
image / occupancy orthographic map is based at least in part on object tracking / track 
classification (see Beymer, Fig. 5, Section 4.2 and 5.2-5.3, the Gaussian mixture model 
image in Fig. 5 is extracted or generated using the occupancy orthographic map image 
in Fig. 5 essentially to assist in tracking and classifying the presence of a human) 

Re Claim 12: Mahbub further discloses said object is a person / occupant (see 
Mahbub, [0081], lines 1-4). Beymer also further discloses said object is a person (see 
Beymer, Figs. 1 and 3, Sections 5.2-5.3). 

Re Claim 1 5: Beymer further discloses said plan-view image / occupancy orthographic 
map image comprises a value based at least in part on a count of pixels / accumulate 
pixels in the corresponding buckets obtained by said visual sensor / camera system and 
associated with said object / person (see Beymer, abstract. Figs. 1 and 3, Sections 4 
Tracking in occupancy maps and 4.1 Occupancy maps, the occupancy orthographic 
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map image is produced by a type of accumulator of pixels in the corresponding 
buckets). 

Re Claim 19: Mahbub further discloses wherein said decision is to distinguish between 
a human / occupant and a non-human / no occupant (see Mahbub, [0081], lines 1-4). 
Beymer also further discloses that track classification determines when a human enters 
[human] and when a human leaves [no human] (see Beymer, Fig. 8, Sections 5.2-5.3). 

Re Claim 20: Mahbub further discloses wherein said decision is to distinguish between 
a plurality of different human body orientations / orientations (see Mahbub, [0126]- 
[0127]). Beymer also further discloses that track classification determines tracking a 
person's location and movement over time [essentially body orientation] (see Beymer, 
Figs. 7a and 7d, Sections 5.2-5.3). 

Re Claim 21 : Mahbub further discloses wherein said decision is to distinguish between 
a plurality of different human body poses / orientations or actual occupancy (see 
Mahbub, [0081], lines 1-4, [0126]-[0127], the different body poses may be the different 
body orientations or just the presence or no presence of the human occupant). Beymer 
also further discloses that track classification determines when a human enters and 
when a human leaves [essentially different body poses] (see Beymer, Figs. 7a and 7d, 
Fig. 8, Sections 5.2-5.3). 
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Re Claim 22: Mahbub further discloses wherein said decision is to distinguish between 
a plurality of different classes of people / infant, occupant, child (see Mahbub, [0081], 
lines 1-4, the different classes of people may be the different type of humans such as an 
infant, child or adult occupant). Beymer also further discloses that track classification 
determines when a human enters and when a human leaves [essentially different 
classes of people] (see Beymer, Figs. 7a and 7d, Fig. 8, Sections 5.2-5.3). 

As to claim 32, the discussions are addressed with respect to claims 1 and 4-5. 

As to claim 34, the discussions are addressed with respect to claim 3. 

As to claim 35, the discussions are addressed with respect to claim 1 . 

As to claim 36, the discussions are addressed with respect to claim 7. 

As to claim 37, the discussions are addressed with respect to claim 1 . 

As to claim 39, the discussions are addressed with respect to claim 4. 

5. Claims 10, 16, 17, 27, and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Mahbub, as modified by Beymer, in further view of Li et al (US 
2003/01 08244 A1 , as applied in previous Office Action). The teachings of Mahbub as 
modified by Beymer have been discussed above. 

However Mahbub, as modified by Beymer, fails to explicitly suggest that the 
classifier is a support vector machine and that the plan-view template is a vector basis 
obtained by principal component analysis (PCA). 
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Li, as recited in claim 10 . discloses said classifier is a support vector machine / 
SVM's (see Li, page 1, paragraph [0008], lines 18-24, "PCA as they rotate and use the 
SVM's for multi-pose face detection"). 

Li, as recited in claim 16 . discloses said plan-view template / frontal face view is 
represented in terms of a vector basis / SVM's (see Li, page 1, paragraph [0008], lines 
18-24, [0011], lines 4-7). 

Li, as recited in claim 17 . discloses said vector basis is obtained through principal 
component analysis (PCA) (see Li, page 1, paragraph [0008], lines 18-24, "PCA as they 
rotate and use the SVM's"). 

Li, as recited in claim 27 . discloses said classifier is a support vector machine / 
SVM's (see Li, page 1, paragraph [0008], lines 18-24, "PCA as they rotate and use the 
SVM's for multi-pose face detection"). 

Li, as recited in claim 38 . discloses said classifier is a support vector machine / 
SVM's (see Li, page 1, paragraph [0008], lines 18-24, "PCA as they rotate and use the 
SVM's for multi-pose face detection"). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to further modify Mahbub's occupancy sensing system and 
method, as modified by Beymer, using Li's teachings by including the capabilities of 
having the classifier be a support vector machine and the plan-view template being a 
vector basis obtained by principal component analysis (PCA) in order to detect a 
person's face in input images containing either frontal or non-frontal views regardless of 
the scale or illumination conditions associated with the face (see Li, [001 1], lines 4-7). 
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6. Claim 2, 11, 13, 14, 18, 24, 28, and 33 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Mahbub, as modified by Beymer, and further in view of 
Bramblet at a! (US 2004/0017929 A1, as applied in previous Office Action). The 
teachings of Mahbub as modified by Beymer have been discussed above. 
Re Claim 24: However Mahbub, as modified by Beymer, doesn't explicitly suggest 
wherein said visual sensor is also for capturing non-depth data. 

Bramblet discloses wherein said visual sensor is also for capturing non-depth 
data / color information (see Bramblet, Fig. 3B, [0019], [0046], stereo pair of tracking 
cameras are placed overhead of the area of observation and color image analysis is 
used to help distinguish and classify multiple objects in an area of observation). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to further modify Mahbub's system, as modified by Beymer, 
using Bramblet's teachings by including to Mahbub's depth capturing visual sensor [as 
modified by Beymer] Bramblet's color attributes of the image to improve the overhead of 
the area observation classification and tracking of objects (see Bramblet, Fig. 3B, 
[0019], [0046]). 

Re Claim 28: Bramblet further discloses wherein said plan-view image is based in part 
on said non-depth data / color information (see Bramblet, Fig. 3B and 4C, [0019], 
[0046], stereo pair of tracking cameras are placed overhead of the area of observation 
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and color image analysis is used to help distinguish and classify multiple objects in an 
area of observation, [0098], a 3D surface analysis provides an image with different 
colors representing the closeness to the camera system). 

As to claims 2 and 1 1 , the claims are the corresponding method claims to claims 
24 and 28 respectively. The discussions are addressed with regard to claims 24 and 
28. 

Re Claim 13: However Mahbub, as modified by Beymer, doesn't explicitly suggest 
wherein said plan-view image information comprises a value based at least in part on 
an estimate of height of a portion of said object above a surface. 

Bramblet discloses wherein said plan-view image comprises a value based at 
least in part on an estimate of height / closeness to camera of a portion of said object 
above a surface (see Bramblet, Fig. 3B and 4C, [0019], [0046], stereo pair of tracking 
cameras are placed overhead of the area of observation and color image analysis is 
used to help distinguish and classify multiple objects in an area of observation, [0098], a 
3D surface analysis provides an image with different colors representing the closeness 
to the camera system). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to further modify Mahbub's system, as modified by Beymer, 
using Bramblet's teachings by including to Mahbub's depth capturing visual sensor [as 
modified by Beymer] Bramblet's surface analysis of the image to improve the overhead 
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of the area observation classification and tracking of objects (see Bramblet, Fig. 3B and 
4C, [0019], [0046], [0098]). 

Re Claim 18: Bramblet further discloses performing height normalization on said plan- 
view template / 2D image representing closeness through different color representation 
[e.g. Fig. 4C] (see Bramblet, Fig. 3B and 4C, [0019], [0046], [0098], the closeness is 
considered as a height normalization as a result of the person being under the camera 
system, a 3D surface analysis provides an image with different colors representing the 
closeness to the camera system which therefore is dependent upon the height 
normalization). 

Re Claim 14: However Mahbub, as modified by Beymer, doesn't explicitly suggest 
wherein said plan-view image comprises a value based at least in part on color data for 
a portion of said object.. 

Bramblet discloses wherein said plan-view image comprises a value based at 
least in part on color data / color information for a portion of said object (see Bramblet, 
Fig. 3B and 4C, [0019], [0046], stereo pair of tracking cameras are placed overhead of 
the area of observation and color image analysis is used to help distinguish and classify 
multiple objects in an area of observation, [0098], a 3D surface analysis provides an 
image with different colors representing the closeness to the camera system). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to further modify Mahbub's system, as modified by Beymer, 
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using Bramblet's teachings by including to Mahbub's deptli capturing visual sensor [as 
modified by Beymer] Bramblet's color attributes of the image to improve the overhead of 
the area observation classification and tracking of objects (see Bramblet, Fig. 3B, 
[0019], [0046]). 

Re Claim 33: Mahbub, as modified by Beymer, discloses said three-dimensional point 
cloud and said plan-view image as discussed in claim 32 for example. 

However Mahbub, as modified by Beymer, doesn't explicitly suggest wherein 
said images are based at least in part on non-depth data. 

Bramblet discloses wherein said visual sensor is also for capturing non-depth 
data / color information (see Bramblet, Fig. 3B, [0019], [0046], stereo pair of tracking 
cameras are placed overhead of the area of observation and color image analysis is 
used to help distinguish and classify multiple objects in an area of observation). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to further modify Mahbub's system, as modified by Beymer, 
using Bramblet's teachings by including Bramblet's color attributes of the image to the 
visual sensor in order to improve the overhead of the area observation classification and 
tracking of objects (see Bramblet, Fig. 3B, [0019], [0046]). 

Allowable Subject Matter 

7. Although claims 29-31 are very similar in language to claims 5, 7, and 8, claim 8 
however is different when compared to claim 31 because claim 5 [claim 5 is a base 
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claim to claim 7, and claim 7 is a base claim to claim 8] is narrower than claim 29 
resulting in different claim interpretations for the two sets of claims. Therefore, only 
claim 8 [and not claim 31, claim 31 stands rejected] is objected to as being dependent 
upon a rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the base claim and any Intervening claims. 

Similarly, claim 40 is objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Conclusion 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to BERNARD KRASNIC whose telephone number is 
(571 )270-1357. The examiner can normally be reached on Mon. - Thur. and every 
other Friday from 8am - 4pm.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571 ) 272-7453. The fax phone number 
for the organization where this application or proceeding Is assigned Is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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you have questions on access to tine Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Bhavesh M Mehta/ 

Supervisory Patent Examiner, Art Unit 2624 
/Bernard Krasnic/ 
January 28, 2010 



